RFB/II/2001/3

ABSTRACT Expanding mandates and roles for Regional Fishery Bodies (RFB) are beginning to evolve worldwide as a result of recently adopted international instruments including the Code of Conduct for Responsible Fisheries. This has implications for RFB memberships and programmes of activities, in particular their fishery control functions. At the same time, Members of RFB, particularly FAO bodies, are being asked to take on greater financial and implementation responsibilities. The nature of the relationship between Members and RFBs is thus expected to change: the support that RFBs should expect to receive from members, and the services that their members should expect from RFBs, will require improved commitments on both sides. Justifying such commitments, particularly with increasing member budgetary and participation commitments (sourced from government and/or resource users), requires performance evaluation and performance management, in both directions. This paper briefly reviews the range of indicators that might be addressed (depending on the particular circumstances within a region) in performance assessments of RFBs and their clients. Firstly, it briefly looks at this issue from the perspective of members' objectives in constituting and tasking an RFB - the stock, fishery control and other issues. It then discusses the indicators for the evaluation of processes, including the management of RFB functions and work programmes, and the supporting arrangements that members should provide. Given the wide differences between, and the evolving nature of, Regional Fishery Bodies, this paper is offered only as a general review of performance evaluation and performance indicators, without specific recommendations.

INTRODUCTION

"Indicators of sustainable development need to be developed to provide solid bases for decision-making at all levels and to contribute to the self-regulating sustainability of integrated environment and development systems."

Chapter 40.4 of Agenda 21, from the United Nations Conference on Environment and Development, 1992

Background

1. In 1992, FAO stated that international fisheries organisations are likely to perform most effectively when their members (i) have congruent fisheries interests, (ii) have a strong economic interest in fisheries, (iii) have a sharp, coherent and consistent management focus, and (iv) have a capacity and will to implement recommendations and decisions (FAO 1992). This combination of characteristics has rarely been met fully or satisfactorily in any fishery body, sometimes because of conflicting interests and rights, but also because of disparities in capacities between members. The degree to which international fisheries organisations achieve these four desirable characteristics, and hence provide some measure of their success or effectiveness, is an important issue.

2. The roles of Regional Fishery Bodies¹ (RFBs) beyond their original advisory functions evolved gradually during the 20th century according to the research and management needs within their areas of jurisdiction (Griffith 1999), and the capacities and political will of their members to fully participate and cooperate in them. In 1982 the United Nations Convention on the Law of the Sea (UNCLOS) provided additional momentum to these changing roles. UNCLOS articles² explicitly require States to cooperate through regional and sub-regional organisations for straddling, highly migratory, anadromous and catadromous fish stocks and marine mammals. During the following decade it became apparent that these articles provided an insufficient framework for the envisaged cooperation. Thus, two further instruments were developed: the FAO Compliance Agreement and the UN Fish Stocks Agreement³. During the same period, FAO elaborated and adopted the Code of Conduct for Responsible Fisheries⁴.
3. Each of these agreed texts (although the UNFSA and FAOCA have not yet entered into force) provide progressively more detail on the role of Regional Fishery Bodies as enabling mechanisms for conservation and management of living marine resources.
4. A number of processes have also been under way since that time, including:

• Progress towards the establishment of new RFBs e.g. in the South-east Atlantic, West and Central Pacific and the South West Indian Oceans, each of which intend to take these new agreements and the Code into account;
• Evaluation of the role of RFBs through analysis and papers presented to the first meeting of FAO and non-FAO RFBs (FAO 1999c), as recommended by the High-Level Panel of Experts (FAO 1998a);
• RFBs, e.g. NEAFC, NAFO and others, are reviewing their roles and mandates with a view to incorporating the evolving instruments and practices of fisheries management. (Lugten 1998)

Scope of this paper

5. The First Meeting of FAO and non-FAO RFBs reviewed the challenges and opportunities for fisheries governance that international cooperation and agreement now offers. The meeting concluded that regional fishery bodies must measure their success by results in the form of favourable trends in or status of fish stocks and human benefits. In response to this concern, the scope of this paper is to set out a general framework of indicators against which the performance⁵ of RFBs in meeting their mandates and responsibilities might be measured.
6. One of the principal problems in doing this is trying to separate the domains of performance for evaluation, and then defining the appropriate indicators that can be used. In this analysis three performance domains for a RFB are defined;

1. Performance as measured by the state of fishery conditions within the area - how well does the RFB meet its objectives of conservation and management?
2. Performance as measured by the degree to which the RFB fulfils its responsibilities - how well does the RFB function on behalf of its members?
3. Performance of RFB members in meeting their responsibilities to the RFB and assisting it to fulfil its activities on their behalf - how well is the RFB supported?

7. This review of indicators is required because the already changing relationships between Members⁶ and RFBs are likely to evolve more rapidly in the near future. Membership, mandates, programmes of activities and methods of financing to meet expanding national and regional responsibilities will change. For instance, the recognition and adoption of `real interest' in RFB membership and changes in financing commitments may see membership alterations. States without `real interest' may withdraw from RFBs, while States hitherto not members (non-contracting parties) may become legally or morally obliged to join.
8. The tasks of active fisheries management, including inter alia the allocation of fishing opportunities and the adoption of powers of monitoring, control and surveillance, will require expanded commitments by member States to their RFBs. Governments will need to justify these increased commitments in order to obtain the necessary national programme support and finance, and to achieve this should be able to demonstrate competent delivery of the objectives, services and tasks which they have set for their RFBs.
9. The widely different mandates of RFBs and their evolving nature in keeping pace with international legal instruments and emerging practices means that this paper can only assess the general nature of performance evaluation and performance indicators. Indeed, some RFBs have already developed complex and sophisticated evaluation mechanisms, including continuing analysis of the performance of fisheries management systems that apply to their particular circumstances and the requirements of their members (ICES 1999c). In support of continuing evaluation, ICES regularly holds dialogue meetings with Partner Commissions (EC, NEAFC, IBSFC, NASCO) to discuss the needs of its clients and the ways they expect ICES advice to be provided (ICES 1999a).

Approaches to the Development of Sustainable Development Indicators

10. Indicators require measurement. In November 1996, measurement practitioners convened in Bellagio, Italy to review progress since the establishment of the UN Commission for Sustainable Development by UNCED in 1992. The report of that meeting (Hardi and Zdan 1997) contained what have become known as the Bellagio Principles - Guidelines for Practical Assessment of Progress towards Sustainable Development. The debate - arriving at these globally applicable sustainable development principles - has mirrored a similar ongoing debate in fisheries; how do we decide what fisheries sustainability is, how do we measure progress towards it and how do we respond to changing conditions?

11. In summary, the 10 Bellagio Principles of Sustainable Performance Measurements recall the need to:

1. decide on Guiding Visions and Goals and hence objectives;
2. take a Holistic Perspective (including consideration of social, ecological, economic subsystems, their state, direction, rates of change and interactions);
3. consider assessment of progress through Essential Elements (including equity, ecological and economic conditions);
4. ensure Adequate Scope (time, space and history);
5. offer Practical Focus (including development of frameworks, indicators of progress, standardisation of measurements, comparison of targets, reference values, ranges, thresholds and directions of trends);
6. encourage Openness (access to data, assumptions, judgement and uncertainty);
7. promote Effective Communication (clear useful information and indicators, particularly for decision makers);
8. include Broad Participation (representation and linkage at all levels to policy and decision makers);
9. encourage Ongoing Assessment (evaluation and adjustments of goals, frameworks and indicators); and
10. ensure Institutional Capacity (through assignment of responsibilities, capacity for data collection, maintenance and documentation).

12. The Bellagio report makes only passing reference to fisheries, but it is clear from that report that its intentions were to cover the full spectrum of practical progress assessment needs for the pathway to sustainable development. It is also clear that these principles have already been taken into account, in one form or another, in the continuing debate within international fisheries, including the preparation and acceptance of the key normative fisheries document, the Code of Conduct for Responsible Fisheries. Indeed, it has been argued that sustainability indicators for fisheries have been a central concern for fisheries scientists and managers for decades. Fisheries scientists and managers have been using indicators and reference points (or standards of sustainability) for target stocks for a long time, even though the terminology was not in general use. For example, catch rates, stock biomass, recruitment levels, costs, revenues, etc. are criteria conventionally and routinely calculated and standardised by fishery science and they can be used to as a basis for sustainability indicators (Garcia and Staples 1999).

Approaches to the Development of Fishery Performance Indicators

13. Indicators are generally used to describe the state or behaviour of a particular criterion that has been chosen to measure progress towards an objective. The modern era of fisheries management science has concentrated mainly on the development of many numerical indicators related to fish stock and fisheries so that they can be used in analytical and descriptive fishery models. This perhaps narrow although critical focus fitted the circumstances in the third quarter of the 20th century, when the main objective was the rapid escalation of fisheries developments and the key issue was estimating potential yield. Today, because the limits to capture fishery potential now seem to have been reached, conservation and environmental concerns, economic optimisation, equity and governance issues - all important component issues of sustainable development - have been added forcibly to the fisheries agenda, and become recognised objectives of fisheries management. The comprehensive and detailed summary of objectives for responsible fisheries contained in the Code of Conduct has been analysed in terms of indicators by Garcia (2000)⁷.
14. The change in indicators over time (hence advance on or retreat from objectives) can only be understood if this is interpreted against a reference value (more commonly referred to as a reference point) that corresponds to the objective. Much progress has been made towards the formulation of appropriate reference points against which the trajectory of indicator time series can be compared (e.g. Caddy and Mahon, 1995), but these still relate mostly to stocks (biomass, fishing mortality, sustainable yield) and revenues (rent, revenue, economic yield). Some of these have become international standards, e.g. maximum sustainable yield (MSY), which has been established as the primary stock harvesting reference point indicator in international instruments.⁸
15. There is now a very large range of possible reference points that might be used in conventional fisheries management (FAO 2000, annex 5). There are generally three types of stock reference point:

• Target Reference Point (TRP): a point at which the indicator is at a level that reflects the particular objective. Management may prescribe actions (say annual targets) that seek to serially approach the target reference point.
• Limit Reference Point (LRP): a point or critical boundary at which the indicator is at a level beyond which it should not fall/rise because it affects sustainability or triggers undesirable management action.
• Threshold Reference Point (ThRP): where there is uncertainty, this is set to provide an advance warning that the LRP is being approached.

16. However, there is a need to fully develop indicators and reference points for other fisheries issues, perhaps using the above `points' formulations. These would include indicators for fishing capacity and effort, biodiversity (endangered species, bycatch, discards, trophic structure), environment (habitat, pollution, aquatic events), governance (institutional capacity, compliance control) and human issues (food security, fishery dependence, employment, poverty). Garcia and Staples (1999, in press) provide a comprehensive review of concepts and elements for a set of guidelines for sustainable development reference systems and indicators for marine capture fisheries.

17. Following UNCED and the call for environmentally sustainable development, the United Nations Commission for Sustainable Development (CSD), the Organisation for Economic Cooperation and Development (OECD), UN Agencies (e.g. FAO), national governments (e.g. Australia) and institutions (e.g. International Institute for Sustainable Development IISD) have been investigating frameworks for the description and analysis of what sustainable development means and how to monitor it. There are numerous frameworks and methodologies for analysing and developing indicators for monitoring sustainable development (Moldan 1997, Meadows 1998, Bossel 1999).⁹
18. In applying these frameworks to fisheries, FAO has produced technical guidelines¹⁰ that offer methodologies for the development of indicators for sustainable development in marine capture fisheries¹¹. The Sustainable Development Reference System (SDRS) approach recommended by these guidelines makes no claim to a single framework or approach; it is offered as a general introductory tool to the issue, which might be adopted at national, regional or global levels in fisheries management.
19. The International Council for the Exploration of the Sea considers that the evaluation of complete fisheries systems is a strategic issue for the ICES Resource Management Committee (ICES 1999c). At the 1999 ICES Annual Science Conference, the FAO SDRS guidelines were presented at a Theme Session on Sustainability Criteria, which reviewed the issues of frameworks and indicators that include biological, economic, social and institutional issues and their relationship to `ecosystem management' (ICES 2000). The Theme Session reviewed an Australian case study based on SDRS and there were suggestions that ICES might apply the same approach to its own case studies.

PERFORMANCE EVALUATION

General requirement

20. The duty and responsibility of States to cooperate either directly or through RFBs is explicit in the new set of international legal fisheries instruments, the Code of Conduct and the emerging FAO International Plans of Action¹². The fundamental aim of this cooperation - to ensure conservation of fish stocks for sustainable exploitation in the future - implies collective responsibility for this objective. The primary role of RFBs is thus safe stewardship of the fish resources for which it is mandated. Secondarily, the collective role is to promote the peaceful resolution of disputes and conflicts through the application of good governance (Swan 1999).
21. To achieve these, RFBs must promote the integration of the interests of its constituents, provide information that meets their requirements for transparency and accountability and set standards (in particular in information) that can be applied by all. Unfortunately, as noted by the FAO High-Level Panel of External Fisheries Experts (FAO 1998a), the performance of many RFBs are hampered by several factors, including:

• Conservation measures being undermined by non-parties to RFBs through illegal fishing, flags of convenience and use of non-regulation fishing gear;
• Failure by members to provide information on their fisheries;
• Political pressures related to sovereignty and national fishery interests (hence diminished commitment to the cooperative effort) and biased interpretation of conservation recommendations to suit national goals;
• Inadequacy or absence of monitoring, control and surveillance;
• Inadequate links between scientific/technical results and policy/implementation processes;
• Lack of finance to meet mandates; and
• Inadequate activity scheduling to meet the necessary decision-making timetables.

22. Of course, these difficulties are not uniform across all RFBs and their members. Nevertheless, measures/indicators of these issues, which can be used with others from year to year, will enable progress in the amelioration of these problems.

23. In general, indicators offer four uses:

• Monitoring progress towards objectives;
• Predicting future potential problems and likely outcomes;
• Simplifying and harmonizing reporting (Evans 1999), and offering comparability between systems and locations to enable learning; and
• Directly informing policies and programmes for the sustainable development of fisheries and the reversal or avoidance of problems.

Basic principles for an evaluation framework

24. Whatever analytical framework is chosen for the evaluation of fisheries systems (e.g. FAO's SDRS) there are basic principles that should be used¹³, including:

• Systems view: analysis of the objectives and constraints of all systems and subsystems, as exemplified by the Code of Conduct for Responsible Fisheries which is an overall fisheries systems view.
• Policy development and multicriteria evaluation: a modelling framework for conceiving and developing policy options and evaluating system-wide impacts of alternative policy options.
• Fisheries principle sub-systems (and their objectives): evaluation according to sub-systems (dimensions), including environmental/biological (sustainability), social benefits (stability), economic (viability) and governance/administrative (efficiency) sub-systems.
• Performance measurability: understanding the absolute, relative or standardised measures of the current state of the components of sub-systems.
• Inductive approach to problem solving: recognising that evaluation is a decision aid to problem solving, and that it needs to be applied on a case-by-case basis.
• Selection of scale: data and information across the sub-systems need to be incorporated at similar levels of complexity and accuracy.

Performance and quality assurance

25. Member States and their constituting bodies (e.g. FAO Article VI and XIV bodies) have a right to expect a level of services from their RFB commensurate with its mandates. Defining the quality of services is a difficult task but it is a process being increasingly addressed by RFBs and their members as clients of the organisations. As pointed out in the introduction to this paper, performance measurement and evaluation is a two way process. The RFB as an institution needs to establish mechanisms to ensure its work meets standards acceptable to its members and against global norms. At the same time members need to establish mechanisms to ensure their contributions to the proper functioning of the organisation are met. The degrees to which an RFB and its members fulfil their responsibilities and commitments are measures of their respective performances.

International standards for management

26. The adoption of international standards for management is one mechanism by which RFBs can address these issues of quality assurance.

"A Standard is a documented agreement containing technical specifications or other precise criteria to be used consistently as rules, guidelines or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose."

27. This general definition comes from the International Organization for Standards (ISO), which publishes a number of mechanisms and methodologies for quality management and quality assurance through its ISO 9000 (institution/organization management) and ISO14000 (environmental management) series. These are voluntary standards that have been derived from international consensus on good management practice.

28. ISO 9000 is, in fact, a family of standards whose aims are to give organizations guidelines on what constitutes a good management system, which in turn can serve as a framework for continuous improvement. The family includes three quality assurance models - ISO 9001, 9002 and 9003 - against which an organisation can audit its own processes, and through which it may invite its clients to audit the quality system to give them confidence that the organisation is capable of delivering the services they need. Alternatively, they can be used as the basis for conformity assessment by an independent quality system certification body.¹⁴

International standards for information

29. The developing Information Society in which we live is evolving standards all the time. Many of these are, or will become, important to RFBs and their members in that they offer methods of information exchange¹⁵ that are independent of proprietary software or hardware. The development and use of the Internet to construct World Wide Web documents offers RFBs an unprecedented facility to offer accessibility to its work as part of their quality management procedures, hence offering transparency and inclusiveness.
30. Hitherto, much of the contention and tension between parties has related to inadequate or un-timely information availability. Using these information exchange standards, and participating in inter-regional standardisation in other issues, will enable RFBs to offer information and services in improved ways.
31. The Coordinating Working Party (CWP) on Fisheries Statistics has been working for many years to establish standards for fisheries information, including nomenclature and statistical classifications. The report of CWP-18 (FAO 1999f), recognising the growing importance of social, economic, commodity and trade issues, provides a detailed review of these issues, including a review of socio-economic indicators and variables. These have been derived and used in the EU-Eurostat New Chronos database and some of them may be important for consideration by RFBs.
32. In support of global fisheries data acquisition that meets much of the information requirements for capture fisheries, FAO has produced guidelines against a Policy-Objectives-Strategy-Implementation framework that traces the definition and selection of indicators, and the establishment of data collection strategies, methods, management, planning and implementation (FAO 1999h).
33. As the mandates and work programme requirements of RFBs expand it is likely that their efforts to increase the types, quantities and qualities of data will become more demanding, particularly with regard to the environment. Attention to global standards to ensure internal consistency and inter-regional and global comparability will become increasingly important.

The basis of confidence

34. Broadly, the basis for confidence in advice received from RFBs is that it should be right, relevant, responsive and respected. Quality policies of RFBs should incorporate these properties. In the context of fisheries they can be summarised as follows:

• Right: advice should be based on the best information available and most appropriate analysis with uncertainty articulated; and provide reasonable predictive power.
• Relevant: advice should be applicable in a practical way so that it can be implemented in the political context for which it was sought.
• Responsive: advice should respond to clients needs in a flexible and timely manner, but should also be forward looking and proactive to future needs.
• Respected: advice should have credibility with policy and decision-makers (and with other interests) who should be persuaded of this in a number of ways, including:
1. Track record of being correct in the past and an understanding of the efforts made to invest in science and information collection for the future;
2. Transparency and consistency of the processes and information resulting in the advice is demonstrable, including assurances of political or sectoral disinterest in arriving at the advice;
3. Peer review (supporting the disinterest requirement) has been conducted in an appropriate manner. Depending on the nature of the advice and other factors (time and budget constraints), this can be Internal (same scientist - repeat assessment), External or Sequential (external scientists - repeat/accept/reject assessment), Integrated (internal and external scientists collaborate for final assessment);¹⁶
4. Effective Communication of the advice has included placing it in the context of previous advice; adaptations to management approaches or changes to targets; use of clear graphics and descriptions.

Internal evaluation

35. The overall motive for establishing quality standards and for conducting continuing, internal performance assessment and evaluation as part of the routine management of an RFB is clear. The RFB should strive to offer the best possible service to its members. There are many areas to consider and these are dealt with in brief detail in Part 4. In summary, performance, related targets and the indicators to assess them should cover, inter alia, the following:

(a) Establishment of a quality policy

The general framework for a quality policy for an RFB would include:

• Transparency: processes are transparent to the direct participants and to outside bodies; participant responsibilities are clearly defined.
• Responsiveness: processes are adaptable to changing requirements of members, while still ensuring good practice.
• Credibility: standing and reliability of advice should be consistent and cognisant of practical problems and their constraints.
• Independence: advice should be objective and free from special interests.
• Consensus: processes must include mechanisms to achieve internal consensus prior to providing advice.
• Integration: advice should be based on the full range of influences across the environmental, social, economic and institutional dimensions.
• Quality control: procedures should be established to ensure `right first time' approaches, including standardisations and error detection.
• Internal peer review: advice should be reviewed internally by peers who were not involved in the original work.
• External peer review: procedures, methods and advice should be periodically peer reviewed by experts from outside the organisation.

(b) Demonstrating efficiency

Within an RFB's quality objectives one of the most important issues is the timeliness in meeting objectives, targets and deadlines. Holding meetings or convening working groups, for instance to prepare advice on TACs to member's decision-makers, must be conducted to meet the demands and conditions in the fisheries.

(c ) Analytical and technical innovation and development

New approaches, particular in stock assessment science, should be developed or adopted to meet requirements at the earliest opportunity, and investment in forward thinking and planning for new and updated methods should be actively sought and properly justified. This will particularly apply to information management and dissemination technologies, compliance control (VMS etc) and fishery-independent science and surveys.

(d) Implementation of resolutions and decisions by member countries

The outcomes of advice to members should be recorded and analysed, including the degree and timeliness of implementation of collective resolutions and decisions; the status of agreed information supply (timeliness, coverage, accuracy) should be monitored and analysed.

(e) Membership's awareness

Through the provision of news and information to members, the RFB should ensure that members are kept up-to-date with information relevant to their interests and responsibilities as members of the organisation, including the status of international instruments and necessary member State actions to implement them, as appropriate; forthcoming regional, inter-regional, global and special interest meetings; and the collective position of RFB members on issues and interests of the region.

(f) Financial and budgetary performance

As with any organisation with financial contributions from a variety of sources, maintaining financial systems to demonstrate efficiency, effectiveness and value for money to its members is extremely important in maintaining credibility. This is particularly the case with RFBs where member's contributions are based on formulae that reflect their different situations and interests in the fisheries of the region. Similarly, many RFBs receive international and bilateral donor assistance through direct programme support and project funding, which are often tied to specific tasks and must be monitored and accounted for separately. Some RFBs may have revenue collection functions now and more will likely do so in the future, particularly in fisheries where the user-pays principle is applied, i.e. fishers contribute through licence fees, lump sum settlements or other mechanisms to the work required for conservation and management and as resource rent.

External evaluation

36. The results of internal performance evaluation should be transparent and accessible to members. Members of RFBs may wish to extend or externalise evaluation of administrative performance in various ways, perhaps formalising this through independent review or certification bodies (e.g. ISO 9000 certification). In the final analysis, however, members must judge for themselves whether there is sufficient accountability, procedural efficiency and responsiveness, value for money and service provision. Most of the expectations on the day-to-day functioning of RFBs are usually catered for in the financial and administrative regulations established by the council or committee that supervises it. However, specific tasks and projects, particularly for member support programmes, may need periodic or ongoing evaluation, including where the RFB is tasked to seek external funding and external implementation of research and development programmes.

Mechanisms and reporting

37. There is no more difficult task than evaluating the performance of tasks when the objectives and expectations of parties are not well defined, including on issues related to timeliness and content of the services to be rendered. As the complexities of RFBs' tasks increase, the details contained within the convention or arrangement establishing them may not be sufficient to properly define the relationship between it and its members on all issues.

38. Indeed, the different positions of some members may mean that different relationships and expectations between them and the organisation may need to be catered for. There may be insufficient capacities for financial contributions, limited technical capacity to enable adequate participation, or special circumstances (e.g. the timing and conduct of national fisheries where compatible measures are meant to apply) that need to be taken into account. Formal and non-formal agreements that apply under particular circumstances may need to be developed, including memoranda of understanding, contracts and partnerships. For example, ICES has recently concluded individual and different Memoranda of Understanding with NASCO, NEAFC, IBSFC and EU. These now stipulate clear and comprehensive sets of standards that meet each organisation's specific requirements. Similarly, where States are the partners of an RFB, Memoranda of Understanding on the provision of individual country services might provide a mechanism for the relationship against that State's specific needs. This is likely to apply, say, when there are necessary compatible fishery management measures that may need to be applied to only one State in the region. Or, when the capacity-building services required for one State are unique to it.

39. Other mechanisms for evaluating and reporting on performance may include Consultative Committees, or performance evaluation working groups and committees, which would set up or adopt their own performance evaluation methods and standards, or do this through the use of external consultants. Whatever the mechanism, transparency will always be maintained if the results of performance are published through the usual mechanism of annual or ad hoc reports.

PERFORMANCE INDICATORS ON OBJECTIVES

Overview of indicators on objectives

40. The trend towards a widened range of detailed RFB objectives and responsibilities is largely accepted as a consequence of international conservation concerns. Cooperation between states through their RFBs in meeting objectives will require knowledge and understanding of the status of the main dimensions of interest, including environment (resources and ecosystem), social and economic and institutional/governance issues.

41. FAO (2000) describes how the indicators may be developed for each of these dimensions and how a Sustainable Development Reference System can interface with fishery management plans and the fishery system, as in Figure 1. That document provides the technical basis on which RFBs might proceed in establishing an SDRS.¹⁷

Figure 1: Relationship between conventional management schemes and a sustainable development reference system (SDRS) (from FAO 2000).

Goals and Objectives of RFBs

42. Regional Fishery Bodies are mandated by their constitutions and programmes in much the same way as national fishery bodies, but most often without the legal powers for the direct enforcement of compliance. They usually seek to build and maintain consensus as the means to fulfil their duties.¹⁸ They are generally not as concerned with the social and economic dimensions of fisheries, except insofar as their aggregate influences and effects impinge on regional management.
43. Objectives can be classified in different ways, as can be seen in the SDRS methodology in FAO (2000). For simplicity, one method might be to define objectives through a general analysis of what RFBs are meant to do, and then define the various components of those task objectives for which indicators should be sought. Given that the overall goal is conservation and safe stewardship, there are 5 general tasks needed to achieve it.

I) TASK 1: Sustainably manage fisheries within biological and environmental limits

The components of this task include ensuring that (i) an understanding of the resources/stocks is available, including stock assessment, ecosystem biodiversity and trophic structure; and (ii) environmental knowledge is available to enable a holistic ecosystems approach to objectives, including issues related to pollution, habitat and physical variability and events.

The control rules that form the basis of fishery management plans and the indicators used to frame them are widely discussed in the literature, particularly in Caddy and Mahon (1995) and Gabriel and Mace (1999). Environmental issues and their indicators are also comprehensively addressed in other documents. Table 1, which summarises Performance Indicators on Objectives of Regional Fisheries Bodies, is therefore relatively simplistic in these areas; there are so many valid and useful indicators on these issues, and they are more comprehensively described elsewhere.

II) TASK 2: Conduct management and control fisheries according to agreed criteria

This task means conducting continuing management of the fishery throughout the period to which control rules apply (year or season) by monitoring (i) catch and fishing capacity (ii) fishing activity in relation to seasonal or spatial restrictions (iii) impacts on the fishery ecosystem, especially through by-catch and discards. It may also mean conducting MCS, including (i) monitoring of, and acting upon, illegal fishing, (ii) assessing general compliance to regulations such as the submission of logsheet, landing sheet or transhipment information, or on the use of fishing gear.

Continuing (almost daily) indicators are needed for (i) catch against TAC or quota allocation, (ii) catch outside and inside regulated areas or seasons, (iii) fleet characteristics (including technical efficiency), currently active capacity and fishing effort, (iv) rate of discards and bycatch against targets, and (v) illegal sightings, inspections and follow up actions.

III) TASK 3: Manage distribution of benefits

The natural resources benefits from regional management (which may or may not include those in EEZs) are generally allocated in one form or another, unless an overall TAC applies. In some cases by-catch allocations may also apply. RFBs manage the allocation processes, usually by providing information and enabling a committee to establish allocation criteria and make allocation decisions. Maintaining indicators for allocation criteria and decisions may mean obtaining information on member's economic and social dependence on fishing, including maintaining historical catch records.

In some cases, where user-pays principles apply to quota holders (States, companies or fishers), direct revenues may be obtained from licence fees (based on vessel effort or actual catch) and/or lump sum fees. Surplus revenues may be distributed amongst members according to certain formulae and/or according to catch distribution (obtained under Task 2). Indicators for the appropriate setting of such fees will need market and price information.

IV) TASK 4: Maintain good governance

The general characteristics of governance and institutional management will apply. Maintenance of good governance will include (i) ensuring their membership and participation is appropriate through a clear understanding of the nature and degree of `real interest' and reporting any changes to all members, (ii) ensuring transparency and the dissemination of information through quality control indicators (timeliness, coverage, etc), (iii) delivering accountability through various mechanisms, (iv) providing assistance to members e.g. through training programmes, (v) administering conflict and dispute settlement, (vi) maintaining relations with non-contracting parties, and (vii) cooperation with adjacent/global bodies. Maintaining an institutional record of all these things will form part of an RFB's quality policy, and performance indicators in these areas can be developed.

V) TASK 5: Conduct research

The components of this task will include (i) data collection and database maintenance, for which indicators of coverage, availability and accuracy should be continually maintained (often automatically through general IT house-keeping), (ii) development of new methods and indicators, for which an indicator of research investment might indicate research performance, (iii) conduct fishery-independent surveys, the records of which would provide indicators of coverage and accuracy useful to stock assessment.

Table 1. Performance Indicators on the objectives of Regional Fishery Bodies

PERFORMANCE INDICATORS ON FUNCTIONS AND RESPONSIBILITIES

Overview of institutional performance

44. The analysis of RFBs at the objectives level as described in the previous section is one way of evaluating their performance. However, as indicated in the introduction, RFBs and their member States form an institutional system in which the performance of each of the partners in fulfilling its functions may be evaluated. Thus, performance indicators may be developed to assess (i) functions - how well does the RFB fulfil its responsibilities on behalf of its members, and (ii) responsibilities - how well do members meet their commitments to the RFB and assist its activities on their behalf - how well is the RFB supported?

45. Alternatively, one might review the general criteria of governance that would apply at a regional level as in Table 2. To a large extent, any framework of indicators is dependent on the needs of the target audience. In this instance, the criteria and indicators are largely what one would expect if viewed from outside the region, and therefore are only partially what may be assessed internally.

Table 2. Criteria and Indicators of governance at the regional level (FAO 2000)

Criteria	Indicators
Compliance Regime	Incentives to comply with the regional agreement The existence of a compliance regime Effectiveness of the regime The existence of outstanding disagreements Integration of global rules
Property Rights	Existence of well defined and recognised property rights Compatibility with sustainability goals Acceptance of major stakeholders
Transparency and Participation	Participation in the regional agreement Incentives for participation in regional agreements Involvement of major stakeholders in making and applying rules of the game Effective communication between stakeholders Capacity to elicit, receive and use information from all stakeholders
Capacity to Manage	Existence of a regional body with competence to manage Terms of regional agreements implemented Degree to which the regional agreement meets sustainable development objectives Existence of an effective dispute resolution process Resources availability at all levels

Performance indicators on functions of RFBs

46. Evaluating an RFB as a device for its members would look at a range of functions and processes, including for the following 6 general areas. Indicators in many of these areas may be simple value judgements as switches (yes/no), or on a scale (good to bad).

(a) Administration of the collective interest of members

How well does it perform as a forum to manage and materialise the objectives of its members? This includes managing the ways in which members (and observers as appropriate) convene for different purposes (council, plenary, committees, working groups), providing the necessary information and arrangements, ensuring appropriate scheduling and meeting deadlines. How transparent and timely are management audits?

Indicators would include the quality of advanced information (e.g. annual diary of events), responsiveness to requests for ad-hoc meetings and other general management quality indicators.

(b) Information management

How effective is the RFB in data collection and maintenance, processing and distributing of information, and raising awareness of (offering advice on) issues, diagnoses and prescriptions? Does it have a well-defined and achieved information technology and communications policy and code of practice that satisfies security and confidentiality requirements?

Indicators for these issues would include timeliness and accuracy of statistical publishing; relevance and accuracy of summary coverage of news and issues; degree to which the RFB participates in debates on future issues and passes information on to members. Does the IT policy match current similar standards, enabling RFB-member and RFB-other body data transfers and communications to proceed? Has data security been breached, how often? Is the data confidentiality hierarchy adequate to protect appropriate levels of interest?

(c) Institutional capacity development and normative tasks

How progressive and anticipatory is the RFB in the planning and normative roles, through the development of frameworks for action and decisions, and standards and norms?

How quickly does the RFB prepare plans - at members' request, or in anticipation of future needs? Does its normative function produce procedures and methods compatible with global, regional or national standards? Does the RFB have programmes to conduct of in-house training and reduce staff turnover? What proportions of the budget are allocated to research and development, and to training?

(d) Training and support to members

How well does it facilitate raising all members to compatible levels of competence and participation, particularly in regions where disparities exist, including the representation of members' interests in inter-regional and global forums, and for international donor support?

Indicators will include measures of changes to capacities of member countries; numbers of scientists and fishery managers trained; relative proportion of trained scientists and fishery managers; attendance at international forums and transmission of results; success rate in obtaining donor project funds.

(e) Output

How good is it in decision-making and providing appropriate advice and recommendations, including its responsiveness (timing and accuracy)?

Indicators would include the results of comparative analysis of predictions and outcomes (see Brander, 1987); timeliness of advice to suit members' needs and fishery requirements.

(f) Implementation

How effective are its implementation programmes - beyond the advisory function - in the conduct of key activities such as compliance control (MCS) and research.

Indicators may include effectiveness of MCS operations; observer coverage of fleets; relative accuracy of observer data; results from research cost-benefit and cost-effectiveness analysis.

Table 3. Performance Indicators on the Functions of Regional Fishery Bodies

Functions and Components	Indicators
Administration of the Collective Interests of Members
Manage meetings (plenary, committee, working group) including attendance of observers	Meeting management quality
Manage dispute resolution processes	Disputes settled/dispute
Manage peer review	Peer reviews/assessment
Audit management quality	Management quality policy indicators
Information Management
Manage data collection programmes	Index of data coverage (by fleet and over time)
Manage, develop and maintain databases	Accuracy/validity of databases
Distribute data and information (statistics, research results)	Speed of publishing; quality of information
Disseminate relevant news (global and regional) on issues, diagnoses and prescriptions	No. newsletters; No. R&D papers;
Maintain information security and confidentiality	No. data security failures Satisfaction level on confidentiality
Institutional Capacity Development and Normative Tasks
Develop frameworks for action (administration and planning), decision-making and procedures	Speed of preparation; quality of documents
Establish norms and standards	Coverage; global compatibility
Develop analytical methods and technology	R&D investment (%budget)
Retain staff and conduct in-house training	Staff turnover rate; % budget on training
Training and Support to Members
Identification of training needs and conduct training	Proportion training needs addressed and capacity enhanced; No. and % trained scientists and fishery managers
Representation of members interests in other fora (interregional, ocean, global, special interest) or for project development with international donor support	No. conferences etc attended; No. members supported at conferences; No. projects funded per projects proposed
Advice and assistance in the provision of data, stock assessment for national waters, MCS	Time/budget spent on assistance
Output
Decision-making and publication of measures in force	Management quality policy indicators; timeliness of decision-making
Provision of appropriate advice	Results from comparative analysis - prediction/actual
Responsiveness	No. ad hoc requests satisfied per requests made
Implementation
Compliance control	MCS effectiveness (decline illegal fishing); observer coverage and accuracy
Research	Results of relative cost-benefit and cost-effectiveness analysis

Performance indicators on responsibilities of members

47. The constraints on the performance of RFBs may be internal and indicators on the processes above will point to weaknesses and gaps that the RFB should rectify. However, some of the problems may stem from the performance of its members in supporting the work of the RFB on their behalf. In the sense that an RFB has a responsibility to identify constraints on its abilities, where these come from external sources, performance evaluation of its members' duties, responsibilities and commitments will assist in identifying (i) weaknesses in capacity that can be addressed through technical and training assistance and (ii) deficiencies in cooperative will that may need to be brought to the attention of members as a whole.

48. This form of evaluation, and the indicators that may be used to measure it, may be more difficult to implement and should be sensitively and diplomatically handled, recognising where possible the member's constraints. Again, they may be simple value judgements as switches (yes/no), or on a scale (good to bad).

(I) Participation

How good is a member at (i) attending meetings (at an appropriate level of personnel), (ii) offering technical expertise for committees and working groups, and (iii) conducting training, research or MCS on behalf of the RFB?

Indicators might include the proportion of statutory and invited meetings attended by member representatives at the appropriate level and skills; number of training course conducted by member for other members or RFB; amount of research or MCS conducted.

(II) Contributions

How committed is the member to support the RFB according to the financial regulations for (i) the provision of recurrent and capital expenditure, and (ii) the provision of in-kind contributions (vessels, infrastructure, personnel)? In the particular case of the Secretariat's host nation, how well does that member live up to the expectations of the agreement?

Indicators might include financial contributions history by member; in-kind contribution by member; extra-budgetary project funding supplied; number of host country/ RFB disputes.

(III) Information supply

How responsive is the member to requests by the RFB (or other member) to (i) requests for general information, (ii) collection and transmission of fishery data of the agreed type, form, accuracy and timeliness, and (iii) provision of reports as required on the introduction of compatible measures?

Indicators would include timeliness of responses to requests; coverage, accuracy and timeliness of fishery data supply (perhaps measured against UNFSA Minimum Data Standards, or other internal standards); timeliness and quality of national reports on the implementation of compatible measures.

(IV) Implementation

How well does a member undertake agreed tasks to (i) comply with collective decisions, (ii) submit to and accept arbitration, dispute settlement or conflict resolution, (iii) establish and maintain compatible measures, (iv) maintain control of flag vessels, or other vessels (as a port State)?

Indicators would include records of compliance (catch, restricted area/season operations, supply of data, etc); timeliness of establishment of compatible measures; number of disputes, arbitrations and settlements; prosecution of flag vessels per citation; inspections of non-flag vessels and follow up actions.

Table 4. Performance Indicators on the responsibilities of members to their Regional Fishery Bodies

Responsibilities and Components	Indicators
Participation
Attendance at meetings	Proportion of statutory and invited meetings attended
Provision of technical expertise	Relative proportion attendees at working groups
Conduct training, research or MCS on behalf of RFB	Relative proportion attendees on surveys/MCS
Contributions
Supply agreed financial support (capital and recurrent budgets)	Contributions payment history;
Supply agreed in-kind support (vessels, infrastructure, personnel)	In-kind contributions history
Information supply
Response to requests for general information	Timeliness and quality
Collection and supply of agreed data (type, form, accuracy, timeliness)	Timeliness and coverage; scale against minimum standards or regional agreement
Report on measures taken	Timeliness and quality
Implementation
Comply with collective decisions	Record of catch/allocation; No. infringements of areas/seasons; quality data supply
Submit to and accept dispute settlements	No. settlements/dispute
Establish and maintain compatible measures	Timeliness
Maintain control of flag vessels and other vessel (as a port State	Prosecutions of flag vessel per citation; No. inspections non-flag vessels conducted

SUGGESTED ACTION BY THE MEETING

49. The meeting is invited to review the processes outlined for assessing the performance of RFBs and suggest the area(s) where greater emphasis might need to be placed to ensure the fish stocks are managed on a sustainable basis.

References and Bibliography:

APPENDIX 1

List of Tables:

Table 1. Performance indicators on the objectives of Regional Fishery Bodies

Table 2. Criteria and indicators of governance at the regional level (FAO 2000)

Table 3. Performance indicators on the functions of Regional Fishery Bodies

Table 4. Performance indicators on the relationship between members and their Regional Fishery Bodies

List of Figures:

Figure 1: Relationship between conventional management schemes and a sustainable development reference system (SDRS) (from FAO 2000).

Abbreviations:

1

In this and other papers the term Regional Fishery Bodies (RFBs) is used to denote all organisations, commissions, committees and arrangements that contribute to fisheries management at sub-regional, regional and international levels. RFBs are also referred to in the literature as Regional Fishery Organisations (RFOs) or international fisheries organisations.

2

UNCLOS Articles 63-67 and 118.

3

FAO Compliance Agreement (FAOCA) - Agreement to Promote Compliance with International Conservation and Management Measures by Fishing Vessels on the High Seas. 1994. UN Fish Stocks Agreement (UNFSA) - Agreement for the Implementation of the Provisions of the United Nations Convention on the Law of the Sea of 10 December 1982 Relating to the Conservation and Management of Straddling Fish Stocks and Highly Migratory Fish Stocks. 1995.

4

Adopted by the FAO Conference at its Twenty-eight Session, 1995.

5

Performance management is a two-way process, and involves performance evaluation at the levels of the organisation and of its members: (i) the extent to which the objectives of the RFB are being met, (ii) the functions undertaken in meeting those objectives and (iii) the responsibilities of members, i.e. addressing the performance domains mentioned above.

6

Including fishing entities and regional economic integration organisations.

7

Annex 1 of Garcia (1999) tabulates the principles, sub-principles, provisions and criteria of the code together with their related indicators, which are also characterised as vector (series of numbers), switch (Yes/No, etc) or matrix (matrix of numbers, multiple time series)

8

UNCLOS Article 61.3 Conservation of living resources and Code of Conduct for Responsible Fisheries, Code of Conduct for Responsible Fisheries, Section 7.2.1 Management Objectives

9

For a brief review of the available approaches, the work of FAO and linkages to the CSD approach (Federal Planning Bureau of Belgium 1997), see The Development of Indicators of Sustainable Development (ISD) in Fisheries (Paper No: XXXX of this meeting), which includes the CSD indicator methodology sheet for MSY.

10

FAO Technical Guidelines for Responsible Fisheries, No. 8. Indicators for sustainable development in marine capture fisheries, 1999. This was produced following the Australian-FAO Technical Consultation on Sustainability Indicators in Marine Capture Fisheries, January 1999, Sydney. (see reference, FAO 2000)

11

The focus of this document on marine capture fisheries reflects the overwhelming global importance of these fisheries and the need to develop normative processes that enable their evaluation. Much of the methodology it identifies would also apply to freshwater fisheries and to some aspects of aquaculture.

12

IPOA for Reducing Incidental Catch of Seabirds in Longline Fisheries, the IPOA for the Conservation And Management of Sharks, the IPOA for the Management of Fishing Capacity. (see reference, FAO 1999h) Two further IPOAs are in preparation - IPOA for Illegal, Unreported and Unregulated fisheries and IPOA for Status and Trends Reporting in Fisheries.

13

These are drawn from ICES 1999c.

14

The ISO neither checks that organisations meet ISO 9000/14000 conformity, nor checks the independent bodies that offer ISO 9000/14000 Certification. All conformity to these standards is voluntary although, of course, RFB members may collectively decide to abide by them. ISO's standards are used to the extent that people find them useful. ISO 9000 Certification, usually derived from a national standards agency (not ISO), is also voluntary and may be unnecessary if internal procedures for implementing the standards are accepted by RFB clients. Nevertheless, such certification provides independent verification and would generally add to the perceived credibility of conformity to standards.

15

SGML (Standard Generalised Markup Language - ISO 8879) and its extensions, HTML (HyperText Markup Language) and XML (eXtensible Markup Language), offer standards whose primary purpose is electronic publishing and data exchange. PDF (Portable Document Format), RTF (Rich Text Format), ODA (Open Document Architecture and Interchange Format - ISO 8613), PS (PostScript) also offer standards for platform-independent document definition.

16

For recent overviews of peer review in fisheries, see reference, O'Boyle et al 1999.

17

It is summarised in The Development of Indicators of Sustainable Development (ISD) in Fisheries (FI:RFB/2/2001/Inf.7), where it is extended with an example of the approach to the formulation and publication of indicators taken by CSD, including the methodology sheet for MSY. The reader is also referred to FAO (2000) for a fuller exposition on SDRS, and to FAO (1999h) for details on ways to collect the necessary information.

18

Consensus is often explicitly described in RFB conventions, alongside formal dispute mechanisms in case consensus cannot be reached.